Processing circuit analyzing image data and generating final image data

ABSTRACT

A processing circuitry is configured to generate a first analysis result based on a size of a partial area of a target area when the partial area is captured by only one of a first sensor or a second sensor, based on first image data for the target area captured by the first sensor and second image data for the target area captured by the second sensor, and generate first final image data or second final image data by using the first image data and the second image data, based on the first analysis result. A difference between the first final image data and the second final image data is based on a difference between a first characteristic of the first sensor and a second characteristic of the second sensor.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of, and claims the benefit under 35U.S.C. § 120 to, U.S. application Ser. No. 16/587,825, filed on Sep. 30,2019, which claims priority under 35 U.S.C. § 119 to Korean PatentApplication No. 10-2019-0008892 filed on Jan. 23, 2019, in the KoreanIntellectual Property Office, the disclosure of which is incorporated byreference herein in its entirety.

BACKGROUND

Some example embodiments described herein relate to devices that receiveimage data from a plurality of sensors.

Various types of electronic devices are being used. An electronicdevice, which includes an image sensor, from among electronic devices isused to capture a subject. For example, an electronic device thatincludes an image sensor may be implemented with one of various types ofelectronic devices such as a smartphone, a tablet personal computer(PC), a laptop PC, a wearable device, etc.

An electronic device may include one image sensor or a plurality ofimage sensors. As imaging technology has advanced, electronic devicesincluding a plurality of image sensors have shown improvements overthose including one image sensor. For example, electronic devicesincluding the plurality of image sensors capture a wider area thanelectronic devices including one image sensor, thus displaying an imageof higher definition.

However, when a plurality of image sensors are mounted in one electronicdevice, the plurality of image sensors are inevitably spaced from eachother. This causes a difference between subjects captured by thedifferent image sensors.

SUMMARY

Some example embodiments provide a processor that selects a way togenerate final image data in consideration of an occlusion area.

According to some example embodiments, a processing circuitry isconfigured to generate a first analysis result based on a size of apartial area of a target area when the partial area is captured by onlyone of a first sensor or a second sensor, based on first image data forthe target area captured by the first sensor and second image data forthe target area captured by the second sensor, and generate first finalimage data or second final image data by using the first image data andthe second image data, based on the first analysis result. A differencebetween the first final image data and the second final image data isbased on a difference between a first characteristic of the first sensorand a second characteristic of the second sensor.

According to some example embodiments, a processor is configured toexecute an instruction stored in a memory, wherein the instruction, whenexecuted by the processor, causes the processor to generate a firstanalysis result based on a size of a partial area of a target area whenthe partial area is included in only one of first image data for thetarget area captured at a first position or second image data capturedat a second position different from the first position. The processor isconfigured to generate first final image data by mapping a portion ofthe first image data onto the second image data or generate second finalimage data by mapping a portion of the second image data onto the firstimage data based on the first analysis result.

According to some example embodiments, a processing circuitry isconfigured to generate an analysis result based on first image data fora first area captured at a first position and second image data for asecond area captured at a second position different from the firstposition, the second area including a portion of the first area. Theprocessing circuitry is configured to generate first final image data bymapping first partial data of the first image data onto the second imagedata or generate second final image data mapping second partial data ofthe second image data onto the first image data, based on the analysisresult.

BRIEF DESCRIPTION OF THE FIGURES

The above and other objects and/or features will become apparent bydescribing in detail some example embodiments with reference to theaccompanying drawings.

FIG. 1 is a block diagram illustrating an exemplary configuration of anelectronic device including a processor according to some exampleembodiments.

FIG. 2 is a conceptual diagram illustrating information included in animage signal output from an image processing block of FIG. 1 .

FIG. 3 is a conceptual diagram illustrating information included in animage signal output from an image processing block of FIG. 1 .

FIG. 4 is a conceptual diagram for describing an operation in which aprocessor of FIG. 1 selects a color transfer method or a detail transfermethod.

FIG. 5 is a conceptual diagram for describing an exemplary operation inwhich a processor of FIG. 1 selects a color transfer method or a detailtransfer method.

FIG. 6 is a block diagram illustrating an exemplary configuration of aprocessor of FIG. 1 .

FIG. 7 is a conceptual diagram for describing a final image generated byusing a detail transfer method.

FIG. 8 is a conceptual diagram for describing a final image generated byusing a color transfer method.

FIG. 9 is a block diagram illustrating an exemplary configuration of anelectronic device including a processor according to some exampleembodiments.

FIG. 10 is a conceptual diagram for describing an exemplary operation inwhich a processor of FIG. 9 selects a color transfer method or a detailtransfer method.

FIG. 11 is a conceptual diagram for describing a final image generatedby using a color transfer method.

FIG. 12 is a flowchart for describing a method in which a processorselects a transfer method.

FIG. 13 is a flowchart for describing a method in which a processorselects a transfer method.

FIG. 14 is a block diagram illustrating an exemplary configuration of anelectronic device including a processor according to some exampleembodiments.

FIG. 15 is a conceptual diagram illustrating information included in animage signal output from an image processing block of FIG. 14 .

FIG. 16 is a flowchart for describing a method in which a processorselects a transfer method.

FIG. 17 is a block diagram illustrating a configuration of an electronicsystem including a processor and interfaces thereof, according to someexample embodiments.

DETAILED DESCRIPTION

Below, some example embodiments may be described in detail and clearlyto such an extent that an ordinary one in the art easily implements thesame.

FIG. 1 is a block diagram illustrating an exemplary configuration of anelectronic device including a processor according to some exampleembodiments.

An electronic device 1000 may include image processing blocks 100 and200 and a processor 300. The electronic device 1000 may capture areas1110, 1120, and 1130 by using the image processing blocks 100 and 200.The electronic device 1000 may display an image associated with theareas 1110, 1120, and 1130 by using the image processing blocks 100 and200 and the processor 300. Subjects 11, 12, 13, 14, and 15 present inthe areas 1110, 1120, and 1130 may be displayed in the image. However,the image may display colors, brightness, etc. of the areas 1110, 1120,and 1130 as well as the subjects 11, 12, 13, 14, and 15. For example,the electronic device 1000 may be implemented with one of various typesof electronic devices such as a smartphone, a tablet personal computer(PC), a laptop PC, an e-book reader, an MP3 player, a wearable device,etc.

The image processing block 100 may include a lens 110, an aperture 120,a sensor 130, and/or an image signal processor 140.

The lens 110 may receive a light reflected from the subjects 11, 12, 13,and 14 present in the areas 1110 and 1120.

The aperture 120 may adjust the amount of light passing through the lens110. The brightness of the image associated with the areas 1110 and 1120may vary with the amount of light passing through the lens 110.

The sensor 130 may be used to capture the areas 1110 and 1120 based onthe light received through the lens 110. The sensor 130 may generate animage signal s1 associated with the areas 1110 and 1120 based on thelight received through the lens 110. The image signal s1 may be anelectrical signal.

The sensor 130 may include a color filter. For example, the color filtermay be an RGB filter (e.g., representatively, having a Bayer pattern).The sensor 130 may receive only a light of a specific wavelength foreach area (e.g., pixel) of the sensor 130 based on the color filter. Forexample, a light of a wavelength ranging from 580 nm to 670 nm may bereceived in an area, in which an “R” filter is included, from amongareas of the sensor 130. The sensor 130 may be used to obtaininformation about a color of the areas 1110 and 1120 based on the lightof the specific wavelength received through the color filter. That is,the sensor 130 may be provided for functions of obtaining informationabout luminance and/or a color of the areas 1110 and 1120.

The sensor 130 may generate the image signal s1 including informationabout elements of the areas 1110 and 1120 based on the light receivedthrough the color filter. For example, the image signal s1 may includeinformation about the luminance of the areas 1110 and 1120 and/orinformation about the color of the areas 1110 and 1120.

The image signal processor 140 may process the image signal s1 togenerate data for the elements of the areas 1110 and 1120. The imagesignal processor 140 may process the image signal s1 to generate datafor the luminance of the areas 1110 and 1120 and/or data for the colorof the areas 1110 and 1120. However, some example embodiments are notlimited thereto. For example, the image signal processor 140 may processthe image signal s1 to generate RGB data and/or to generate YCbCr data.The image signal processor 140 may output an image signal s2 includingthe data for the luminance of the areas 1110 and 1120 and/or the datafor the color of the areas 1110 and 1120.

The image processing block 200 may include a lens 210, an aperture 220,a sensor 230, and/or an image signal processor 240.

The lens 210 may receive a light reflected from the subjects 12, 13, 14,and 15 present in the areas 1120 and 1130.

The aperture 220 may adjust the amount of light passing through the lens210. The brightness of the image associated with the areas 1120 and 1130may vary with the amount of light passing through the lens 210.

The sensor 230 may generate an image signal s3 based on the lightreceived through the lens 210. The image signal s3 may be an electricalsignal.

The sensor 230 may be used to obtain information about an element of theareas 1120 and 1130 based on the light received through the lens 210.That is, the sensor 230 may be provided for a function of obtaininginformation about luminance of the areas 1120 and 1130.

The sensor 230 may generate the image signal s3 including informationabout the element of the areas 1120 and 1130 based on the light receivedthrough the lens 210. The image signal s3 may include information aboutthe luminance of the areas 1120 and 1130.

Characteristics of the sensor 130 and the sensor 230 may becomplementary. For example, the sensor 130 and the sensor 230 may beimplemented with sensors of different types. In this case, the sensor130 and the sensor 230 may have different functions. The sensor 130 maybe a sensor including a color filter, and/or the sensor 230 may be ahigh-sensitivity sensor.

For another example, the sensor 130 and the sensor 230 may beimplemented with sensors of the same type or of similar types, but mayhave functions of different performance. According to some exampleembodiments, either or both of the sensor 130 and/or the sensor 230 maybe implemented using one or more photodiodes and/or any other type ofsensor capable of generating the image signals s1 and s3, as would beunderstood by a person having ordinary skill in the art. Both the sensor130 and the sensor 230 may include a color filter, but sensitivity ofthe sensor 230 may be higher than sensitivity of the sensor 130. Also,both the sensor 130 and the sensor 230 may include a color filter, butthe sensor 130 may have a wider field of view than the sensor 230. Forexample, the sensor 130 may capture the area 1110, 1120, 1130, while thesensor 230 may capture the area 1120, 1130. Complementarycharacteristics of the sensor 130 and the sensor 230 may be variouslycombined and used. In detail, the characteristics of the sensor 130 andthe sensor 230 may be combined in a way to indicate the area captured bythe sensor 130 more minutely (e.g., in greater detail). Also, thecharacteristics of the sensor 130 and the sensor 230 may be combined ina way to indicate the area captured by the sensor 230 more minutely. Inthe following descriptions, it is assumed that the sensor 130 is asensor including a color filter and the sensor 230 is a high-sensitivesensor that does not include a color filter, but some exampleembodiments are not limited thereto.

The image signal s3 may not include information about a color of theareas 1120 and 1130. However, the sensor 230 may receive lights of allwavelengths without blocking a light of a specific wavelength.Accordingly, the amount of light that the sensor 230 receives may begreater than the amount of light that the sensor 130 receives at thesame time or contemporaneously. In the case where the sensor 130includes an RGB color filter, the amount of light that the sensor 230receives may be three times greater than the amount of light that thesensor 130 receives at the same time or contemporaneously. Accordingly,with regard to the luminance of the area 1120, the image signal s3 mayinclude more accurate information than the image signal s1.

The lens 210 and the sensor 230 may be spaced from the lens 110 and thesensor 130. That is, the lens 210 and the sensor 230 may be spaced fromthe lens 110 and the sensor 130 as much as a specific distance. As aresult, the areas 1110 and 1120 captured through the sensor 130 may bedifferent from the areas 1120 and 1130 captured through the sensor 230.The sensor 130 may capture the area 1110, but the sensor 230 may notcapture the area 1110. The sensor 230 may capture the area 1130, but thesensor 230 may not capture the area 1110. According to some exampleembodiments, the area 1110 may include the area 1130.

The sensors 130 and 230 may capture the area 1120 in common. However,the sensor 130 may capture the area 1120 at a place where the sensor 130is located, and the sensor 230 may capture the area 1120 at a placewhere the sensor 230 is located. As a result, pieces of informationabout the subjects 12, 13, and 14 included in the image signals s1 ands3 may be different from each other.

For example, the image signal s1 may include information about thesubjects 12 and 13 and may not include information about the subject 14.Because the subject 14 is covered by the subject 13 when viewed from theplace where the sensor 130 is located, the subject 14 may not becaptured by the sensor 130.

For another example, the image signal s3 may include information aboutthe subjects 13 and 14 and may not include information about the subject12. Because the subject 12 is covered by the subject 13 when viewed fromthe place where the sensor 230 is located, the subject 12 may not becaptured by the sensor 230.

The image signal processor 240 may process the image signal s3 togenerate data for the element of the areas 1120 and 1130. For example,the image signal processor 240 may process the image signal s3 togenerate data for the luminance of the areas 1120 and 1130. The imagesignal processor 240 may output an image signal s4 including the datafor the luminance of the areas 1120 and 1130.

Unlike the image signal s2, the image signal s4 may not include data fora color. However, with regard to the luminance of the area 1120, theimage signal s4 may include more accurate information than the imagesignal s2.

The processor 300 may perform various operations for controlling overalloperations of the electronic device 1000. The processor 300 may controloperations of the image processing block 100 and/or may process theimage signals s2 and s4 received from the image processing block 100.According to some example embodiments, operations described herein asbeing performed by any or all of the image processing block 100, theimage processing block 200, the image signal processor 140, the imagesignal processor 240, the processor 300 and/or the electronic device1000 may be performed by processing circuitry. The term ‘processingcircuitry,’ as used in the present disclosure, may refer to, forexample, hardware including logic circuits; a hardware/softwarecombination such as a processor executing software; or a combinationthereof. For example, the processing circuitry more specifically mayinclude, but is not limited to, a central processing unit (CPU), anarithmetic logic unit (ALU), a digital signal processor, amicrocomputer, a field programmable gate array (FPGA), a System-on-Chip(SoC), a programmable logic unit, a microprocessor, application-specificintegrated circuit (ASIC), etc. For example, the processor 300 may beimplemented with a general-purpose processor, a special-purposeprocessor, or an application processor, and may include one or moreprocessor cores. However, some example embodiments are not limitedthereto. For example, the processor 300 and/or the image signalprocessors 140 and 240 may be included in one complementary metal-oxidesemiconductor image sensor (CIS) chip. The processor 300 may be oneimage signal processor that receives the signals s2 and s4 to generatefinal image data. In this case, the processor 300 may output the finalimage data to a processor such as a general-purpose processor, aspecial-purpose processor, or an application processor.

A memory 330 may temporarily store data (e.g., data processed and/or tobe processed by the processor 300) used for an operation of theelectronic device 1000. For example, the memory 330 may include avolatile and/or nonvolatile memory such as a static random access memory(SRAM), a dynamic RAM (DRAM), a synchronous DRAM (SDRAM), a phase-changeRAM (PRAM), a magneto-resistive RAM (MRAM), a resistive RAM (ReRAM),and/or a ferro-electric RAM (FRAM).

The processor 300 may control operations of the image processing block100. For example, the processor 300 may control the operations of theimage processing blocks 100 and 200 based on a control value that theuser inputs through an input interface of the electronic device 1000.For another example, the processor 300 may analyze signals received fromthe image processing blocks 100 and 200 to control the operations of theimage processing block 100.

The processor 300 may adjust a distance between the lenses 110 and 210and the sensors 130 and 230. The processor 300 may adjust the distancebetween the lenses 110 and 210 and the sensors 130 and 230 such that thesubject 13 captured in common by the sensors 130 and 230 is recognizedby the sensors 130 and 230 as having the same size or similar sizes.

The processor 300 may analyze the signals received from the imageprocessing blocks 100 and 200 to calculate a distance from the imageprocessing blocks 100 and 200 to the areas 1110, 1120, and 1130. Forexample, the image processing blocks 100 and 200 may include one or moreof a distance sensor, a photo diode, an infrared light receivingelement, a charge coupled device, and/or a piezoelectric element. Forexample, the processor 300 may calculate the distance from the imageprocessing blocks 100 and 200 to the areas 1110, 1120, and 1130 by usinga triangulation manner and/or an ultrasonic manner.

The processor 300 may adjust a position of the sensors 130 and 230and/or the lenses 110 and 210, based on the calculated distance. Theprocessor 300 may adjust the position of the sensors 130 and 230 and/orthe lenses 110 and 210 to adjust a focus between the lenses 110 and 210and the sensors 130 and 230.

In the following descriptions, it is assumed that a distance from theimage processing blocks 100 and 200 to the areas 1110, 1120, and 1130calculated by the processor 300 is calculated on the basis of thesensors 130 and 230. That is, it is assumed that the distance from theimage processing blocks 100 and 200 to the areas 1110, 1120, and 1130 isa distance d1 from the sensors 130 and 230 to the areas 1110, 1120, and1130.

The processor 300 may output data for the calculated distance d1 to thememory 330. The memory 330 may store the data of the calculated distanced1 received from the processor 300.

The processor 300 may adjust openings of the apertures 120 and 220. Theprocessor 300 may adjust a shutter speed. The processor 300 may controlthe amount of light incident to the sensors 130 and 230 by adjusting theopenings of the apertures 120 and 220 and/or the shutter speed. Thesensors 130 and 230 may sense brightness of the areas 1110, 1120, and1130. The processor 300 may receive a signal associated with the sensedbrightness from the image processing blocks 100 and 200. The processor300 may adjust the openings of the apertures 120 and 220 and/or theshutter speed based on the sensed brightness.

The processor 300 may output data for a level of the sensed brightnessto the memory 330. The memory 330 may store the data of the level of thesensed brightness received from the processor 300.

The processor 300 may process the image signals s2 and s4 received fromthe image processing blocks 100 and 200 to generate a final imagesignal. The final image signal may be used to display an image in anoutput interface (e.g., a display device) of the electronic device 1000.However, some example embodiments are not limited thereto. For example,the processor 300 may process three or more image signals to generate afinal image signal. In this case, the processor 300 may generate thefinal image signal through operations that are the same or substantiallythe same as operations to be described below. According to some exampleembodiments, the electronic device 1000 may include a third imageprocessing block having a third sensor and configured to generate thirdimage data based on an image signal generated by the third sensor. Theprocessor 300 may generates a final image signal based on image signalss2, s4 and the image signal generated by the third sensor. For example,the processor 300 may generate the final image signal based on a size ofa region (e.g., an occlusion region) captured by only one or two of thefirst sensor, the second sensor or the third sensor.

The image signal s2 may include information about a color, luminance,etc. of the areas 1110 and 1120. Also, the image signal s2 may includeinformation about an outline, a shape, a size, a color, etc. of thesubjects 11, 12, and 13. The image signal s4 may include informationabout a color, luminance, etc. of the areas 1120 and 1130. Also, theimage signal s4 may include information about an outline, a shape, asize, a color, etc. of the subjects 13, 14, and 15.

The processor 300 may analyze the image signals s2 and s4 to generate afinal image signal in various methods. The processor 300 may generate afinal image by using a fusion method. The fusion method refers to amethod of selecting one of a color transfer (CT) manner and/or a detailtransfer (DT) manner based on a result of analyzing the image signals s2and/or s4 and generating a final image based on the selected manner.Below, to select a transfer method means to select the color transfermethod or the detail transfer method. However, some example embodimentsare not limited thereto. For example, the processor 300 may generatefinal image data by using a first manner or a second manner, based on aresult of analyzing the image signals s2 and s4. A difference betweenfinal image data generated in the first manner and final image datagenerated in the second manner may be based on a difference betweencharacteristics of the sensors 130 and 230.

Also, data for the distance d1 from the image processing blocks 100 and200 to the areas 1110, 1120, and 1130 may be stored in the memory 330.The processor 300 may select a transfer method by using data stored inthe memory 330. That is, the processor 300 may select a transfer methodfor generating a final image by using the result of analyzing the imagesignals s2 and s4, the data for the distance d1, and/or the data for thelevel of the brightness.

An operation in which the processor 300 analyzes the image signals s2and s4 to select a color transfer method or a detail transfer methodwill be more fully described with reference to FIGS. 2 to 16 .

In the case where the color transfer method is selected, the processor300 may generate a final image signal based on the image signal s4 anddata for color included in the image signal s2. In this case, an outputimage that is displayed based on the final image signal may include theareas 1120 and 1130.

In the case where the detail transfer method is selected, the processor300 may generate a final image signal based on the image signal s2 anddata for luminance included in the image signal s4. In this case, anoutput image that is displayed based on the final image signal mayinclude the areas 1110 and 1120. The color transfer method and thedetail transfer method will be more fully described with reference toFIGS. 4 and 5 .

FIG. 2 is a conceptual diagram illustrating information included in animage signal output from an image processing block of FIG. 1 .

An image 1140 is an example of an image displayed based on the imagesignal s2 output from the image processing block 100. In the case wherethe area 1120 is captured at a place where the sensor 130 is located,because the subject 14 is covered by the subject 13, the subject 14 maynot be captured by the sensor 130. Accordingly as described withreference to FIG. 1 , the image signal s1 may not include informationabout the subject 14. Also, the image signal s2 may not includeinformation about the subject 14. The image 1140 may not include thesubject 14.

However, some example embodiments are not limited thereto. For example,a portion of a subject may cover another portion of the subject. Also,one subject may cover a plurality of subjects, and/or one subject maycover a portion of any other subject.

Below, an area that is covered by the subject 13 included in the image1140 is referred to as an “occlusion area”. The subject 14 may be in theocclusion area. As the distance d1 from the sensor 130 to the areas 1110and 1120 decreases, the size of the occlusion area may become larger. Asthe distance d1 from the sensor 130 to the areas 1110 and 1120increases, the size of the occlusion area may become smaller. FIGS. 1 to8 are associated with the case where the distance d1 from the sensor 130to the areas 1110 and 1120 is shorter, and the case where the distanced1 from the sensor 130 to the areas 1110 and 1120 is longer will bedescribed with reference to FIGS. 9 to 13 .

The image signal s2 may include data for colors and/or luminance of theareas 1110 and 1120. The image signal s2 may include data for colorsand/or luminance of the subjects 11, 12, and 13 located in the areas1110 and 1120.

Colors of the areas 1110 and 1120 may be displayed in the image 1140.Colors of the subjects 11, 12, and 13 may be displayed in the image1140. Patterns displayed in the image 1140 may indicate colors of thesubjects 11, 12, and 13, and different patterns may indicate differentcolors.

As described with reference to FIG. 1 , due to a color filter, theamount of light that the sensor 130 receives may be smaller than theamount of light that the sensor 230 receives at the same time orcontemporaneously. In detail, each pixel of the sensor 130 may receive alight corresponding to only one color (e.g., one of “R”, “G”, and/or“B”) by a color filter (e.g., via a color filter). Accordingly, in theimage signal processor 140, interpolation utilizing color data of anadjacent pixel may be used to generate data for luminance of the areas1110 and 1120. As a result, a resolution of the image 1140 that isdisplayed based on the image signal s2 may be lower than a resolution ofan image that is displayed based on the image signal s4.

Also, an ISO gain (e.g., an analog gain and/or a digital gain) of thesensor 130 may be set to be higher than an ISO gain of the sensor 230.Accordingly, the image signal processor 140 may generate data forluminance (indicating brightness) that is similar to data for luminancegenerated by the image signal processor 240. However, for this reason,noise included in the image 1140 that is displayed based on the imagesignal s2 may be greater than noise included in an image that isdisplayed based on the image signal s4, and thus, a signal-to-noiseratio (SNR) may decrease. Also, a resolution of the image signal s2 maybe lower than a resolution of the image signal s4.

Accordingly, an outline, a surface, a pattern, etc. of the subjects 11,12, and 13 may not be minutely represented (e.g., represented in greaterdetail) in the image 1140. For example, in the case where the subject 12is a person, a nose, eyes, a mouth, etc. of the subject 12 may not berepresented.

FIG. 3 is a conceptual diagram illustrating information included in animage signal output from an image processing block of FIG. 1 .

An image 1150 is an example of an image displayed based on the imagesignal s4 output from the image processing block 200. In the case wherethe area 1120 is captured at a place where the sensor 230 is located,because the subject 12 is covered by the subject 13, the subject 12 maynot be captured by the sensor 230. Accordingly as described withreference to FIG. 1 , the image signal s3 may not include informationabout the subject 12. Also, the image signal s4 may not includeinformation about the subject 12. The image 1150 may not include thesubject 12.

The image signal s4 may include data for luminance of the areas 1120 and1130. The image signal s4 may include data for luminance of the subjects13, 14, and 15 located in the areas 1120 and 1130.

As described with reference to FIG. 1 , the amount of light that thesensor 230 receives may be greater than the amount of light that thesensor 130 receives at the same time or contemporaneously. In detail,each pixel of the sensor 230 may receive a light of the whole visiblespectrum. Accordingly, the image signal processor 240 may not useseparate interpolation for generating data for luminance of the areas1120 and 1130. As a result, a resolution of the image 1150 that isdisplayed based on the image signal s4 may be higher than a resolutionof the image 1140 that is displayed based on the image signal s2.

Also, because light sensitivity of the sensor 230 is higher than lightsensitivity of the sensor 130, an ISO gain of the sensor 230 may be setto be lower than an ISO gain of the sensor 130. As a result, noiseincluded in the image signal s4 may be smaller than noise included inthe image signal s2. Also, for this reason, a resolution of the image1150 that is displayed based on the image signal s4 may be higher than aresolution of the image 1140 that is displayed based on the image signals2.

Accordingly, an outline, a surface, a pattern, etc. of the subjects 13,14, and 15 may be minutely represented (e.g., represented in greaterdetail) in the image 1150. Referring to the image 1140 illustrated inFIG. 2 , the subject 13 that is displayed in the image 1150 may beclearer than the subject 13 that is displayed in the image 1140.

However, as described with reference to FIG. 1 , the image signal s4 maynot include data for colors of the areas 1120 and 1130. Accordingly, theareas 1120 and 1130 included in the image 1150 may be displayed by ablack color and/or a white color (e.g., a gray level). The subjects 13,14, and 15 included in the image 1150 may be displayed by a black colorand/or a white color (e.g., a gray level).

FIG. 4 is a conceptual diagram for describing an operation in which aprocessor of FIG. 1 selects a color transfer method and/or a detailtransfer method. For better understanding, FIGS. 1 and 4 will bereferenced together.

The processor 300 may compare image blocks based on the image signal s2and image blocks based on the image signal s4, by using the imagesignals s2 and s4. Each of the image blocks based on the image signal s2may include information about one or more of pixels constituting theimage 1140. Each of the image blocks based on the image signal s4 mayinclude information about one or more of pixels constituting the image1150. According to some example embodiments, the size and/or positioningof the image blocks of the image signals s2 and/or s4 may be designparameters determined through empirical study.

The processor 300 may compare the image blocks based on the image signals2 and the image blocks based on the image signal s4 to detect imageblocks 1152 corresponding to an occlusion area (also referred to as aregion and/or a portion of an image herein). As described with referenceto FIG. 2 , the occlusion area may be an area that is covered by thesubject 13 illustrated in FIG. 1 and thus is not displayed in the image1140. In detail, the occlusion area may be an area that is displayed inthe image 1150 but is not displayed in the image 1140 due to the subject13. The image blocks 1152 indicating the occlusion area may notcorrespond to the image blocks based on the image signal s2. Accordingto some example embodiments, the image blocks 1152 indicating theocclusion area may include blocks of which at least a threshold portioncorresponds to the occlusion area. The threshold portion may be a designparameter determined through empirical study. The processor 300 may draw(e.g., determine) a size of the occlusion area from the number (e.g.,quantity) of the detected image blocks 1152 and/or a size of thedetected image blocks 1152. In the following descriptions, the size ofthe image blocks 1152 may correspond to the number (e.g., quantity) ofpixels that the image blocks 1152 include. Accordingly, the larger thesize of the image blocks 1152 the greater the number of pixels that theimage blocks 1152 includes. Also, the size of the occlusion area may beexpressed by the number of pixels included in the occlusion area.

However, some example embodiments are not limited thereto. For example,the occlusion area may include an area that is covered by the subject 13and thus is not displayed in the image 1150 and/or an area that is notdisplayed in the image due to the subject 13. The occlusion area may bean area, which is relatively large, from among the area that is coveredby the subject 13 and thus is not displayed in the image 1150 and thearea that is not displayed in the image due to a subject (e.g., imageblock 1141).

The processor 300 may draw the size of the occlusion area by using anoptical flow technique, a stereo matching technique, etc., but someexample embodiments are not limited thereto. In the case of using theoptical flow technique, the processor 300 may draw the size of theocclusion area based on a magnitude of a change in a motion vector. Forexample, as the occlusion area increases, the change in the motionvector may increase.

In the case of using the stereo matching technique, the processor 300may generate a depth map and may draw the size of the occlusion areabased on a change in a depth. For example, a large occlusion area may bedetected based on an area where a depth sharply changes.

The processor 300 may select a transfer method based on a size of thedetected occlusion area and/or the distance d1 from the sensors 130and/or 230 to the areas 1110, 1120, and/or 1130.

The processor 300 may select a transfer method based on a result ofcomparing the distance d1 and a reference distance.

The processor 300 may select a transfer method based on a result ofcomparing the size of the detected occlusion area and a reference size.The processor 300 may select a transfer method based on a result ofcomparing the number of pixels included in the detected occlusion areaand a reference count.

The processor 300 may select a transfer method based on a result ofcomparing the number of image blocks (or the number of pixels) includedin the image blocks 1152 and the reference count. An operation ofcomparing the number of image blocks and the reference count may be thesame as or substantially the same as an operation of comparing the sizeof the detected occlusion area and the reference size. Accordingly, inthe following descriptions, the operation of comparing the size of thedetected occlusion area and the reference size may be replaced with theoperation of comparing the number of image blocks and the referencecount.

The processor 300 may select a transfer method by considering the sizeof the detected occlusion area and the distance d1 in various manners.

For example, when the distance d1 is smaller than the referencedistance, the processor 300 may select the detail transfer method. Whenthe distance d1 is greater than the reference distance, the processor300 may select the color transfer method or the detail transfer methodbased on the size of the occlusion area. In this case, when the size ofthe detected occlusion area is greater than the reference size, theprocessor 300 may select the detail transfer method. When the size ofthe detected occlusion area is smaller than the reference size, theprocessor 300 may select the color transfer method.

For another example, the processor 300 may select the color transfermethod and/or the detail transfer method by converting the distance d1and the size of the detected occlusion area and comparing the generatedconversion value and a reference value. According to some exampleembodiments, any or all of the reference distance, the reference size,the reference count, and/or the reference value may be design parametersdetermined through empirical study.

The processor 300 may select a transfer method in various methods.However, as a result, the processor 300 may select the detail transfermethod when the size of the detected occlusion area is great and mayselect the color transfer method when the size of the detected occlusionarea is small.

The processor 300 may select a transfer method in consideration of boththe size of the detected occlusion area and the distance d1, thusreducing the amount of data to be processed and improving reliability ofa transfer method selected by the processor 300.

The processor 300 may compare the image blocks based on the image signals2 and the image blocks based on the image signal s4 to detect the imageblock 1141 corresponding to an image block 1151. The image block 1141may include information that corresponds to or is similar to informationthat the image block 1151 includes. For example, the image blocks 1141and 1151 may include information about luminance of a portion of thesubject 13. For another example, the image block 1141 may includeinformation about a color of a portion of the subject 13.

In the case of selecting the color transfer method, the processor 300may map data for color included in the image signal s2 onto the imagesignal s4. The processor 300 may map the data for color included in theimage block 1141 onto the image block 1151 corresponding to the imageblock 1141. In the case where the data for color included in the imageblock 1141 is mapped onto the image block 1151, data corresponding tothe image block 1151 may include the data for color included in theimage block 1141. Accordingly, a color may be displayed in an image thatis based on the data corresponding to the image block 1151.

However, because the image blocks 1152 do not correspond to image blocksbased on the image signal s2, data for color included in the imagesignal s2 may not be mapped onto the image blocks 1152.

In the case of selecting the detail transfer method, the processor 300may map data for luminance included in the image signal s4 onto theimage signal s2. The processor 300 may map the data for luminanceincluded in the image block 1151 onto the image block 1141 correspondingto the image block 1151.

For example, in the case where the data for luminance included in theimage block 1151 is mapped onto the image block 1141, data for luminanceincluded in the image block 1141 may be replaced with the data forluminance included in the image block 1151. In this case, the processor300 may generate a final image signal by using the data for luminanceincluded in the image block 1151 and the data for color included in theimage block 1141.

For another example, in the case where the data for luminance includedin the image block 1151 is mapped onto the image block 1141, data forluminance included in the image block 1141 may include (e.g., may beadded to, combined with, etc.) the data for luminance included in theimage block 1151. In this case, the processor 300 may generate a finalimage signal by using the data for luminance included in the image block1151, the data for luminance included in the image block 1141, and thedata for color included in the image block 1141.

FIG. 5 is a conceptual diagram for describing an exemplary operation inwhich a processor of FIG. 1 selects a color transfer method and/or adetail transfer method. For better understanding, FIGS. 1 and 5 will bereferenced together.

The processor 300 may select the color transfer method and/or the detailtransfer method based on a size of an area 1155 (also referred to as aregion and/or a portion of an image herein) not displayed in both theimage 1140 and the image 1150 (e.g., displayed in image 1150 and not inimage 1140, and/or displayed in image 1140 and not in image 1150) and/orthe distance d1 from the sensors 130 and/or 230 to the areas 1110, 1120,and/or 1130.

As in the way described with reference to FIG. 4 , the processor 300 maydetect the area 1155. However, a method in which the processor 300selects a transfer method based on a size of an occlusion area isdescribed with reference to FIG. 4 , and a method in which the processor300 selects a transfer method based on a size of the area 1155 will bedescribed with reference to FIG. 5 .

The size of the area 1155 may be determined based on the size of theocclusion area and a size of the areas 1110 and/or 1130. That is, thesize of the area 1155 may correspond to a sum of the size of the area1110 or the area 1130 and the size of the occlusion area.

The size of the areas 1110 and 1130 may vary with a distance between thesensors 130 and 230 and the distance d1 from the sensors 130 and 230 tothe areas 1110, 1120, and 1130. Because the distance between the sensors130 and 230 is fixed, the size of the areas 1110 and 1130 may vary withthe distance d1. As the distance d1 decreases, the size of the areas1110 and 1130 may increase compared with the size of the area 1120.

The processor 300 may select the color transfer method and/or the detailtransfer method based on the size of the area 1155 and the distance d1from the sensors 130 and 230 to the areas 1110, 1120, and 1130. Asdescribed with reference to FIG. 4 , the processor 300 may select atransfer method by considering the size of the area 1155 and thedistance d1 in various manners.

For example, when the distance d1 is smaller than the referencedistance, the processor 300 may select the detail transfer method. Whenthe distance d1 is greater than the reference distance, the processor300 may select the color transfer method or the detail transfer methodbased on the size of the area 1155. In detail, when the size of the area1155 is greater than the reference size, the processor 300 may selectthe detail transfer method. In detail, when the size of the area 1155 issmaller than the reference size, the processor 300 may select the colortransfer method.

For another example, the processor 300 may select the color transfermethod and/or the detail transfer method by converting the distance d1and the size of the area 1155 and comparing the generated conversionvalue and a reference value.

The processor 300 may select a transfer method in various methods.However, as a result, the processor 300 may select the detail transfermethod when the size of the area 1155 is great and may select the colortransfer method when the size of the area 1155 is small.

FIG. 6 is a block diagram illustrating an exemplary configuration of aprocessor of FIG. 1 .

The processor 300 may include an analyzing circuitry 310 and agenerating circuitry 320. Each of the analyzing circuitry 310 and thegenerating circuitry 320 may perform some of operations that theprocessor 300 provides. According to some example embodiments,operations described herein as being performed by either or both of theanalyzing circuitry 310 and/or the generating circuitry 320 may beperformed by processing circuitry.

In detail, the analyzing circuitry 310 may receive the image signals s2and s4. By using the image signals s2 and s4, the analyzing circuitry310 may perform an operation of drawing (e.g., determining) a size of anocclusion area and/or an operation of drawing (e.g., determining) thedistance d1 from the sensors 130 and 230 to the areas 1110, 1120, and1130. Also, the analyzing circuitry 310 may perform an operation ofcomparing the size of the occlusion area and a reference size and/or anoperation of comparing the distance d1 and a reference distance. Theanalyzing circuitry 310 may output a signal g0 based on a result of thecomparison.

The generating circuitry 320 may receive the signal g0. The signal g0may indicate a result of comparing the size of the occlusion area andthe reference size and/or a result of comparing the distance d1 and thereference distance. The generating circuitry 320 may select the colortransfer method or the detail transfer method based on the signal g0.The generating circuitry 320 may generate a final image signal by usingthe selected transfer method. The analyzing circuitry 310 and thegenerating circuitry 320 are exemplified to describe configurations andoperations of the processor 300, but some example embodiments are notlimited thereto.

FIG. 7 is a conceptual diagram for describing a final image generated byusing a detail transfer method. FIG. 8 is a conceptual diagram fordescribing a final image generated by using a color transfer method. Forbetter understanding, FIGS. 7 and 8 will be referenced together.

An image 1160 is an example of an image displayed based on a final imagesignal that is generated by using the detail transfer method. Asdescribed with reference to FIG. 4 , in the case where the image signals2 does not include data corresponding to the area 1130 that dataincluded in the image signal s4 includes, the image 1160 may not displaythe area 1130. Accordingly, the image 1160 that is generated by usingthe detail transfer method may be associated with the areas 1110 and1120. Also, the subject 13 included in the image 1160 may be biased tothe right.

As described with reference to FIG. 1 , noise included in the imagesignal s4 may be smaller than noise included in the image signal s2.Accordingly, a resolution of the image 1160 may be higher than aresolution of the image 1140 illustrated in FIG. 2 . Also, an outputimage signal may include less noise than the image signal s2. That is,the image 1160 may be clearer than the image 1140. Also, the subject 13included in the image 1160 may be more minute (e.g., detailed) than thesubject 13 included in the image 1140.

An image 1170 is an example of an image displayed based on a final imagesignal that is generated by using the color transfer method. Asdescribed with reference to FIG. 4 , in the case where the image signals4 does not include data corresponding to the area 1110 that dataincluded in the image signal s2 includes, the image 1170 may not includethe area 1110. Accordingly, the image 1170 that is generated by usingthe color transfer method may be associated with the areas 1120 and1130. Also, the subject 13 included in the image 1170 may be biased tothe left.

Accordingly, the user may determine whether a final image is generatedin any transfer method of the detail transfer method and/or the colortransfer method, through areas included in a final image and/or aposition of a subject included in the final image.

As described with reference to FIG. 1 , a color of the image 1170 may bedisplayed based on data for a color included in the image signal s2.However, because the image signal s2 does not include color dataassociated with the subjects 14 and 15, the subjects 14 and 15 of theimage 1170 may be displayed by a black color and/or a white color. As anocclusion area or the area 1155 illustrated in FIG. 5 becomes greater,the appearance of color loss phenomenon and/or artifacts may increase.Also, colors of the subjects 14 and 15 included in the image 1170 may bedifferent from actual colors. As an occlusion area or the area 1155illustrated in FIG. 5 becomes greater, the appearance of this pseudoloss phenomenon may appear increase.

Accordingly, in the case where an occlusion area is great, the processor300 may generate a final image signal by using the detail transfermethod, thus preventing or reducing the color loss phenomenon, theartifacts, and/or the pseudo loss phenomenon. Also, the processor 300may generate the final image signal based on the image signal s2 anddata for luminance included in the image signal s4, thus improving thequality of the final image.

FIG. 9 is a block diagram illustrating an exemplary configuration of anelectronic device including a processor according to some exampleembodiments. For better understanding, FIGS. 1 and 9 will be referencedtogether.

FIG. 9 shows the case where the electronic device 1000 is locatedfurther from the areas 1110, 1120, and 1130 of FIG. 1 than theelectronic device 1000 illustrated in FIG. 1 . In the followingdescriptions, it is assumed that a distance d2 from the sensors 130and/or 230 to the areas 1110, 1120, and/or 1130 is greater than thedistance d1 illustrated in FIG. 1 .

As the distance d2 from the sensors 130 and 230 to the areas 1110, 1120,and 1130 increases, a size of an area 1120 a, which is captured incommon by the sensors 130 and 230, may increase. All the subjects 11,12, 13, 14, and 15 may be in the area 1120 a captured in common by thesensors 130 and 230. An area 1110 a that is captured only by the sensor130 and an area 1130 a that is captured only by the sensor 230 maydecrease in size.

In this case, as a result, an occlusion area or the area 1155illustrated in FIG. 5 may decrease, and the processor 300 may output afinal image signal by using the color transfer method.

As described with reference to FIG. 1 , an image signal s5 may includedata for colors and/or luminance of the areas 1110 a and 1120 a. Theimage signal processor 140 may generate data for the luminance of theareas 1110 a and 1120 a and/or data for the color of the areas 1110 aand 1120 a, based on the image signal s5. An image signal s6 may includedata for the luminance of the areas 1110 a and 1120 a and/or data forthe color of the areas 1110 a and 1120 a.

The image signal s7 may include data for colors and/or luminance of theareas 1120 a and 1130 a. The image signal processor 140 may generatedata for the luminance of the areas 1120 a and 1130 a, based on theimage signal s7. An image signal s8 may include data for luminance ofthe areas 1120 a and 1130 a. With regard to the luminance of the area1120 a, the image signal s8 may include more accurate information thanthe image signal s6.

FIG. 10 is a conceptual diagram for describing an exemplary operation inwhich a processor of FIG. 9 selects a color transfer method and/or adetail transfer method.

An image 1140 a is an example of an image displayed based on the imagesignal s6 output from an image processing block 110. An image 1150 a isan example of an image displayed based on the image signal s8 outputfrom an image processing block 200.

As described with reference to FIG. 4 , the processor 300 may compareimage blocks based on the image signal s6 and image blocks based on theimage signal s8 to detect an image block(s) 1152 a corresponding to anocclusion area.

As described with reference to FIG. 4 , the processor 300 may draw(e.g., determine) a size of the occlusion area from the number of thedetected image block(s) 1152 a and/or a size of the detected imageblock(s) 1152 a. In the case a distance of the sensors 130 and/or 230from the areas 1110 a, 1120 a, and/or 1130 a increases, the size of theocclusion area may decrease. For example, the image blocks 1152illustrated in FIG. 4 include four image blocks; in contrast, the imageblock 1152 a may include one image block.

That the size of the occlusion area varies with a distance from theareas 1110, 1120, and/or 1130 to the sensors 130 and/or 230 may beunderstood from a result of comparing FIG. 4 and FIG. 10 . In detail, asthe distance from the areas 1110, 1120, and/or 1130 to the sensors 130and/or 230 decreases, the size of the occlusion area may increase. Asthe distance from the areas 1110, 1120, and/or 1130 to the sensors 130and/or 230 increases, the size of the occlusion area may decrease, orthe occlusion area may disappear.

However, as described with reference to FIG. 4 , the occlusion area maybe variously exemplified. That is, an image block including an occlusionarea may mean an image block 1142 a, an image block 1152 a, and/or theimage blocks 1142 a and/or 1152 a.

FIG. 11 is a conceptual diagram for describing a final image generatedby using a color transfer method. For better understanding, FIGS. 9 and11 will be referenced together.

In the case where a distance of the sensors 130 and/or 230 from theareas 1110, 1120, and/or 1130 increases as illustrated in FIG. 9 and/orin the case where a size of an occlusion area decreases as illustratedin FIG. 10 , the processor 300 may generate a final image by using thecolor transfer method.

An image 1170 a is an example of an image displayed based on a finalimage signal that is generated by using the color transfer method. Thata color loss phenomenon and/or a pseudo color phenomenon decreases ordisappears may be understood from a result of comparing the image 1170 awith the image 1170 illustrated in FIG. 8 .

The amount of data for luminance included in the signal s8 may begreater than the amount of data for color included in the signal s6.Accordingly, in the case of using the color transfer method, the amountof data to be processed by the processor 300 may be smaller than in thecase of using the detail transfer method. Also, a speed at which theprocessor 300 generates an output image signal in the case of using thecolor transfer method may be higher than in the case of using the detailtransfer method.

That is, the processor 300 may select a transfer method appropriatelybased on the distance d2 and/or the size of the occlusion area, thusdecreasing the amount of data to be processed. Also, as the artifacts,the color loss phenomenon, and/or the pseudo color phenomenon decreases,the electronic device 1000 may display a final image having a highresolution.

FIG. 12 is a flowchart for describing a method in which a processorselects a transfer method. For better understanding, FIGS. 1 and 12 willbe referenced together.

In operation S510, the sensors 130 and 230 may capture the areas 1110,1120, and 1130. The sensor 130 may capture the areas 1110 and 1120 at aplace where the sensor 130 is located. The sensor 230 may capture theareas 1120 and 1130 at a place where the sensor 230 is located.

In operation S520, the processor 300 may compare the distance d1 fromthe sensors 130 and/or 230 to the areas 1110, 1120, and/or 1130 with areference distance. The processor 300 may calculate the distance d1 andmay output data for the distance d1 to the memory 330. The memory 330may store the data of the distance d1 thus calculated. The processor 300may use the data for the distance d1 stored in the memory 330 forcomparing the distance d1 with the reference distance.

When the distance d1 is greater than or equal to the reference distance,in operation S530, the processor 300 may compare image blocks that arebased on the image signals s2 and s4. The processor 300 may detect imageblocks indicating an occlusion area by comparing image blocks. Theprocessor 300 may draw (e.g., determine) a size of the occlusion areafrom the number of the detected image blocks and/or a size of thedetected image blocks.

In operation S540, the processor 300 may compare the size of theocclusion area with a reference size.

When the size of the occlusion area is smaller than the reference size,in operation S550, the processor 300 may generate a final image signalby using the color transfer method.

When the size of the occlusion area is greater than or equal to thereference size, in operation S560, the processor 300 may generate afinal image signal by using the detail transfer method.

When the distance d1 is smaller than the reference distance, inoperation S560, the processor 300 may generate a final image signal byusing the detail transfer method. In operation S570, the electronicdevice 1000 may display an image based on the final image signal.

However, some example embodiments are not limited thereto. For example,when the distance d1 is greater than or equal to the reference distance,in operation S560, the processor 300 may generate a final image signalby using the detail transfer method. Also, when the distance d1 issmaller than the reference distance, operation S530 may be performed,and then, in operation S540, the processor 300 may select the detailtransfer method or the color transfer method.

In the case where operation S560 is performed after operation S520 isperformed, the processor 300 may not calculate the size of the occlusionarea. Accordingly, as the processor 300 performs an operation ofcomparing the distance d1 and the reference distance prior to anoperation of comparing the size of the occlusion area and the referencesize prior, the amount of data to be processed by the processor 300 maydecrease.

FIG. 13 is a flowchart for describing a method in which a processorselects a transfer method. For better understanding, FIGS. 1 and 13 willbe referenced together.

In operation S610, the sensors 130 and/or 230 may capture the areas1110, 1120, and/or 1130. The sensor 130 may capture the areas 1110 and1120 at a place where the sensor 130 is located. The sensor 230 maycapture the areas 1120 and 1130 at a place where the sensor 230 islocated.

In operation S620, the processor 300 may compare image blocks that arebased on the image signals s2 and s4. The processor 300 may detect imageblocks indicating an occlusion area by comparing image blocks. Theprocessor 300 may draw (e.g., determine) a size of the occlusion areafrom the number of the detected image blocks and/or a size of thedetected image blocks.

In operation S630, the processor 300 may convert the size of theocclusion area thus drawn and the calculated distance d1 to generate aconversion value. The processor 300 may generate the conversion value invarious methods; however, the conversion value may be generated to beproportional to the size of the occlusion area thus drawn and to beinversely proportional to the calculated distance d1, or the conversionvalue may be generated to be inversely proportional to the size of theocclusion area thus drawn and to be proportional to the calculateddistance d1.

In operation S640, the processor 300 may compare the conversion valueand a reference value.

When the conversion value is smaller than the reference value, inoperation S650, the processor 300 may generate a final image signal byusing the color transfer method.

When the conversion value is greater than or equal to the referencevalue, in operation S660, the processor 300 may generate a final imagesignal by using the detail transfer method. However, some exampleembodiments are not limited thereto. For example, when the conversionvalue is greater than or equal to the reference value, the processor 300may generate a final image signal by using the color transfer method.Also, when the conversion value is smaller than the reference value, theprocessor 300 may generate a final image signal by using the detailtransfer method.

In operation S670, the electronic device 1000 may display an image basedon the final image signal.

The user may check whether a method that is used to generate the finalimage signal is changed, based on an image displayed in the electronicdevice 1000. In detail, in the case where a distance from the sensors130 and/or 230 to the areas 1110, 1120, and/or 1130 changes from thedistance d1 to the distance d2, referring to FIGS. 7 and 11 , an area tobe displayed in a final image may change. Also, a position of thesubject 13 included in the final image may be shifted from the right tothe left (or from left to right). As such, the user may check that(e.g., whether) a method that is used to generate a final image signalis changed. Also, the user may determine whether the method that is usedto generate the final image signal is the color transfer method or thedetail transfer method.

FIG. 14 is a block diagram illustrating an exemplary configuration of anelectronic device including a processor according to some exampleembodiments. For better understanding, FIGS. 1 and 14 will be referencedtogether.

FIG. 14 shows areas 1110 b, 1120 b, and 1130 b, the illuminance of whichis lower than that of the areas 1110, 1120, and 1130. FIG. 14 isassociated with the case where the electronic device 1000 captures theareas 1110 b, 1120 b, and 1130 b of low illuminance.

As described with reference to FIG. 1 , the processor 300 may adjustopenings of the apertures 120 and 220. The processor 300 may adjust ashutter speed. The processor 300 may control the amount of lightincident to the sensors 130 and 230 by adjusting the openings of theapertures 120 and 220 and/or the shutter speed.

The sensors 130 and/or 230 may sense brightness of the areas 1110 b,1120 b, and/or 1130 b. The processor 300 may receive a signal associatedwith the sensed brightness from the image processing blocks 100 and/or200. The processor 300 may adjust the openings of the apertures 120and/or 220 and the shutter speed based on the sensed brightness.

In the case where the electronic device 1000 captures the areas 1110 b,1120 b, and 1130 b of low illuminance, brightness levels of the areas1110 b, 1120 b, and/or 1130 b that the processor 300 senses may be low(e.g., lower than a threshold illuminance level, also referred to hereinas a performance reference level). To appropriately maintain the amountof light incident to the sensors 130 and/or 230, based on the sensedbrightness, the processor 300 may make the openings of the apertures 120and/or 220 wide and/or may make the shutter speed slow.

Nevertheless, in the case where the electronic device 1000 captures theareas 1110 b, 1120 b, and/or 1130 b of low illuminance or very lowilluminance, the amount of light incident to the sensors 130 and 230 maybe relatively small. Also, a difference between the amount of light thatthe sensor 130 receives and the amount of light that the sensor 230receives may be great (e.g., lower than a threshold difference level).According to some example embodiments, the threshold illuminance leveland/or the threshold difference level may be design parametersdetermined through empirical study.

The sensors 130 and 230 may generate signals s9 and s11 based on theincident light. Magnitudes of the signals s9 and s11 may vary with theamount of incident light. The image signal processor 140 may generatesignals s10 and s12 based on the signals s9 and s11. The signals s10 ands12 may include information about the magnitudes of the signals s9 ands11, respectively.

Accordingly, the processor 300 may determine that the areas 1110 b, 1120b, and/or 1130 b are areas of low illuminance, based on the signals s10and/or s12. In detail, the processor 300 may determine that the areas1110 b, 1120 b, and/or 1130 b are areas of low illuminance, based on adifference between the amount of data for luminance included in thesignal s10 and/or the amount of data for luminance included in thesignal s12 (e.g., based on whether the first performance of the sensor130 is higher or lower than the second performance of the sensor 230).Also, the processor 300 may determine that the areas 1110 b, 1120 b,and/or 1130 b are areas of low illuminance, based on the sensedbrightness. Returning to FIG. 6 , the analyzing circuitry 310 maydetermine that the areas 1110 b, 1120 b, and/or 1130 b are areas of lowilluminance, based on the signals s10 and/or s12. The analyzingcircuitry 310 may output the signal g0. The signal g0 may indicatewhether the areas 1110 b, 1120 b, and/or 1130 b are areas of lowilluminance. The generating circuitry 320 may select a transfer methodbased on the signal g0. The generating circuitry 320 may generate afinal image signal by using the selected transfer method.

FIG. 15 is a conceptual diagram illustrating information included in animage signal output from an image processing block of FIG. 14 .

An image 1140 b is an example of an image displayed based on the imagesignal s10 output from the image processing block 110. In the case wherethe sensor 130 captures the areas 1110 b and 1120 b of low illuminance,colors of the subjects 11, 12, and 13 may not be displayed in the image1140 b. In this case, to generate a final image signal, it may be moreimportant to protect luminance data included in the image signal s12than to protect color data included in the image signal s10.

In the case where the detail transfer method is used, in the process ofmapping luminance data included in the image signal s12 onto the imagesignal s10, the luminance data may be lost, and/or noise may occur inthe luminance data. Accordingly, in the case where the areas 1110 b,1120 b, and/or 1130 b are determined as an area of low illuminance, theprocessor 300 may generate a final image signal by using the colortransfer method.

FIG. 16 is a flowchart for describing a method in which a processorselects a transfer method. For better understanding, FIGS. 14 and 16will be referenced together.

In operation S710, the sensors 130 and/or 230 may capture the areas 1110b, 1120 b, and/or 1130 b. The sensor 130 may capture the areas 1110 band 1120 b at a place where the sensor 130 is located. The sensor 230may capture the areas 1120 b and 1130 b at a place where the sensor 230is located.

In operation S720, the processor 300 may compare brightness levels ofthe areas 1110 b, 1120 b, and/or 1130 b with a reference level (e.g.,threshold illuminance level, performance reference level, etc.). Asdescribed with reference to FIGS. 1 and 14 , the processor 300 may sensebrightness of the areas 1110 b, 1120 b, and/or 1130 b. The processor 300may output data for a level of the sensed brightness to the memory 330.The memory 330 may store the data for the level of the sensedbrightness. The processor 300 may use the data for the brightness levelstored in the memory 330 for comparing the level of the sensedbrightness with the reference level.

When the level of the sensed brightness is greater than or equal to thereference level, in operation S730, the processor 300 may compare imageblocks that are based on the image signals s10 and s12. The processor300 may detect image blocks indicating an occlusion area by comparingimage blocks. The processor 300 may draw (e.g., determine) a size of theocclusion area from the number of the detected image blocks and/or asize of the detected image blocks.

In operation S740, the processor 300 may compare the size of theocclusion area with a reference size.

When the size of the occlusion area is greater than the reference size,in operation S750, the processor 300 may generate a final image signalby using the detail transfer method.

When the size of the occlusion area is smaller than or equal to thereference size, in operation S760, the processor 300 may generate afinal image signal by using the color transfer method.

When the level of the sensed brightness is smaller than the referencelevel, in operation S760, the processor 300 may generate a final imagesignal by using the color transfer method.

In operation S770, the electronic device 1000 may display an image basedon the final image signal. As in the method described with reference toFIG. 13 , the user may check whether a method that is used to generatethe final image signal is changed, based on an image displayed in theelectronic device 1000.

FIG. 17 is a block diagram illustrating a configuration of an electronicsystem including processing circuitry and interfaces thereof, accordingto some example embodiments. An electronic system 2000 may beimplemented with a data processing device capable of using or supportingan interface offered by mobile industry processor interface (MIPI)alliance. For example, the electronic system 2000 may be implementedwith at least one of electronic devices, such as a digital camera, avideo camcorder, a smartphone, a tablet, a wearable device (e.g., asmart watch, a smart band, and/or the like), etc.

The electronic system 2000 may include processing circuitry (e.g., anapplication processor 2100), a display 2220, and/or an image sensors2230 and 2231. The application processor 2100 may include a DigRF master2110, a display serial interface (DSI) host 2120, a camera serialinterface (CSI) host 2130, and/or a physical layer 2140.

The DSI host 2120 may communicate with a DSI device 2225 of the display2220 in compliance with the DSI. For example, an optical serializer SERmay be implemented in the DSI host 2120. For example, an opticaldeserializer DES may be implemented in the DSI device 2225.

The CSI host 2130 may communicate with CSI devices 2235 and 2236 ofimage sensors 2230 and 2231 through the CSI. For example, an opticaldeserializer may be implemented in the CSI host 2130. For example, anoptical serializer SER may be implemented in the CSI devices 2235 and2236.

For example, when the electronic system 2000 is implemented with anelectronic device (e.g., a digital camera or a video camcorder) which isable to capture an image, each of the image sensors 2230 and 2231 maycapture an image of an object to generate an image signal (refer to FIG.1 ). The application processor 2100 may generate a final image signalbased on image signals respectively output from the image sensors 2230and 2231. The electronic system 2000 may display a final image in thedisplay 2220 based on the final image signal.

The electronic system 2000 may further include a radio frequency (RF)chip 2240 that communicates with the application processor 2100. The RFchip 2240 may include a physical layer 2242, a DigRF slave 2244, and/oran antenna 2246. For example, the physical layer 2242 of the RF chip2240 and the physical layer 2140 of the application processor 2100 mayexchange data with each other through a DigRF interface supported by theMIPI alliance.

The electronic system 2000 may further include a working memory 2250and/or embedded/card storage 2255. The working memory 2250 and/or theembedded/card storage 2255 may store data received from the applicationprocessor 2100.

The working memory 2250 may temporarily store data processed and/or tobe processed by the application processor 2100. The working memory 2250may include a volatile memory such as an SRAM, a DRAM, and/or an SDRAM,and/or a nonvolatile memory such as a flash memory, a PRAM, an MRAM, aReRAM, and/or a FRAM.

The embedded/card storage 2255 may store data regardless of whether apower is supplied. The embedded/card storage 2255 may include one ormore nonvolatile memories, a memory controller, and/or a buffer.

The electronic system 2000 may communicate with an external systemthrough a communication module, such as a worldwide interoperability formicrowave access (WiMAX) 2260, a wireless local area network (WLAN)2262, and/or an ultra-wideband (UWB) 2264, and/or the like. Even thoughthe WiMAX 2260, the WLAN 2262 and the UWB 2264 are mentioned, theelectronic system 2000 may further include various communicationmodules.

The electronic system 2000 may further include a speaker 2270 and/or amicrophone 2275 for processing voice information. The electronic system2000 may further include a global positioning system (GPS) device 2280for processing position information. The electronic system 2000 mayfurther include a bridge chip 2290 for managing connections betweenperipheral devices.

A processor of some example embodiments may select a color transfermethod and/or a detail transfer method in consideration of an occlusionarea, an illuminance environment, and/or a distance between a subjectand an electronic device.

According to some example embodiments, as a transfer method isappropriately selected, the amount of data to be processed by theprocessor may decrease, and/or the quality of final image may beimproved.

While some example embodiments have been described with reference toexamples thereof, it will be apparent to those of ordinary skill in theart that various changes and modifications may be made thereto withoutdeparting from the spirit and scope of some example embodiments as setforth in the following claims.

What is claimed is:
 1. An electronic device, comprising: a first imageprocessing block configured to capture a target area, and generate firstimage data including color information of the target area; a secondimage processing block configured to capture the target area, andgenerate second image data including distance information, the distanceinformation corresponding to a distance between the target area and aposition from which the target area is captured; and a processorconfigured to generate a first analysis result based on the distanceinformation, generate a second analysis result based on a level ofbrightness of the target area, and generate first final image data orsecond final image data based on the first analysis result and thesecond analysis result, the first final image data or the second finalimage data being generated by using the first image data and the secondimage data.
 2. The electronic device of claim 1, wherein the secondimage processing block is configured obtain luminance information of thetarget area; and the second image data includes the luminanceinformation.
 3. The electronic device of claim 2, wherein the processoris configured to: select a color transfer manner or a detail transfermanner based on the first analysis result and the second analysisresult; and perform at least one of generating the first final imagedata based on the color information and the second image data by usingthe color transfer manner; or generating the second final image databased on the luminance information and the first image data by using thedetail transfer manner.
 4. The electronic device of claim 3, wherein theprocessor is configured to select the detail transfer manner in responseto the distance corresponding to the distance information being shorterthan a reference distance.
 5. The electronic device of claim 3, whereinthe processor is configured to select the color transfer manner inresponse to the brightness of the target area being lower than areference level.
 6. The electronic device of claim 1, wherein theprocessor is configured to: generate a third analysis result based on asize of a partial area of the target area, the partial area beingcaptured by only one of the first image processing block or the secondimage processing block; and generate the first final image data or thesecond final image data based on the first analysis result, the secondanalysis result and the third analysis result.
 7. The electronic deviceof claim 6, wherein the processor is configured to: generate the thirdanalysis result by determining whether the size of the partial area issmaller than a reference size.
 8. A method of operating an electronicdevice, the electronic device including a first image processing blockand a second image processing block, and the method comprising:generating, by the first image processing block, first image dataincluding color information of a target area; generating, by the secondimage processing block, second image data including luminanceinformation of the target area; generating a first analysis result basedon distance information, the distance information corresponding to adistance between the target area and a position from which the targetarea is captured; generating a second analysis result based on a levelof brightness of the target area; and generating first final image dataor second final image data based on the first analysis result and thesecond analysis result, the first final image data or the second finalimage data being generated by using the first image data and the secondimage data.
 9. The method of claim 8, wherein the generating the firstimage data includes obtaining, by the first image processing block, thedistance information; the first image data includes the distanceinformation; and the generating the first analysis result includesgenerating the first analysis result based on the distance informationof the first image data.
 10. The method of claim 8, wherein thegenerating the second image data includes obtaining, by the second imageprocessing block, the distance information; the second image dataincludes the distance information; and the generating the first analysisresult includes generating the first analysis result based on thedistance information of the second image data.
 11. The method of claim8, further comprising: generating a third analysis result based on asize of a partial area of the target area, the partial area beingcaptured by only one of the first image processing block or the secondimage processing block, wherein the generating the first final imagedata or the second final image data includes generating the first finalimage data or the second final image data based on the first analysisresult, the second analysis result and the third analysis result. 12.The method of claim 8, further comprising: selecting a color transfermanner or a detail transfer manner based on the first analysis resultand the second analysis result; and performing at least one ofgenerating the first final image data based on the color information andthe second image data by using the color transfer manner; or generatingthe second final image data based on the luminance information and thefirst image data by using the detail transfer manner.
 13. The method ofclaim 12, wherein the selecting comprises selecting the detail transfermanner in response to the distance corresponding to the distanceinformation being shorter than a reference distance.
 14. The method ofclaim 12, wherein the selecting comprises selecting the color transfermanner in response to the brightness of the target area being lower thana reference level.
 15. A non-transitory computer-readable medium storinginstructions that, when executed by a processor, cause the processor toperform a method, the method comprising: obtaining first image dataincluding color information of a target area; obtaining second imagedata including luminance information of the target area; generating afirst analysis result based on distance information, the distanceinformation corresponding to a distance between the target area and aposition from which the target area is captured; generating a secondanalysis result based on a level of brightness of the target area; andgenerating first final image data or second final image data based onthe first analysis result and the second analysis result, the firstfinal image data or the second final image data being generated by usingthe first image data and the second image data.
 16. The non-transitorycomputer-readable medium of claim 15, wherein the obtaining the firstimage data includes obtaining the distance information; the first imagedata includes the distance information; and the generating the firstanalysis result includes generating the first analysis result based onthe distance information of the first image data.
 17. The non-transitorycomputer-readable medium of claim 15, wherein the obtaining the secondimage data includes obtaining the distance information; the second imagedata includes the distance information; and the generating the firstanalysis result includes generating the first analysis result based onthe distance information of the second image data.
 18. Thenon-transitory computer-readable medium of claim 15, wherein the methodfurther comprises: generating a third analysis result based on a size ofa partial area of the target area, the partial area being included inonly one of the first image data or the second image data, wherein thegenerating the first final image data or the second final image dataincludes generating the first final image data or the second final imagedata based on the first analysis result, the second analysis result andthe third analysis result.
 19. The non-transitory computer-readablemedium of claim 15, wherein the method further comprises: selecting acolor transfer manner or a detail transfer manner based on the firstanalysis result and the second analysis result; and performing at leastone of generating the first final image data based on the colorinformation and the second image data by using the color transfermanner; or generating the second final image data based on the luminanceinformation and the first image data by using the detail transfermanner.
 20. The non-transitory computer-readable medium of claim 19,wherein the selecting comprises selecting the detail transfer manner inresponse to the distance corresponding to the distance information beingshorter than a reference distance.